1. Field of the Invention
This invention pertains generally to the generation of higher-resolution pictures and video from lower-resolution pictures and videos, and more particularly to controlling artifacts and noise when generating these higher-resolution pictures and video.
2. Description of Related Art
Apparatus and methods for simplifying the creation of high-definition (high-resolution) content have become increasingly sought after by the photographic and more particularly video industries. The increased importance of these mechanisms is especially profound in view of recent changes which make high-definition video content the standard for video streams.
Techniques are being increasingly implemented for creating higher-resolution images from lower-resolution images. These resultant higher-resolution pictures (whether used in still pictures or video), are created by utilizing higher resolution information (such as from fusing together several low-resolution (LR) to form one enhanced-resolution image) to increase the resolution of an input image or image stream within a ‘super-resolution’ (SR) technique. Thus, super-resolution (SR) techniques enhance the resolution of an imaging system. Single-frame and multiple-frame variants of SR techniques exist, with multiple-frame techniques being generally considered the most useful.
However, these existing super-resolution techniques are subject to a number of problems. One shortcoming of SR techniques is that it is very difficult to make tradeoffs between sharpness and artifact elimination caused by ringing. In creating super-resolution images, the high-frequency components are boosted, which not only creates stronger ringing artifacts, but emphasizes the noise component. These ringing problems are noticeable with regard to pictures, yet can be even more pronounced when viewing video. The inability to effectively tradeoff between sharpness and the elimination of ringing artifacts poses a serious challenge for existing techniques.
One common approach to increasing picture resolution, is shown in FIG. 1 wherein an up-scale filtering process is applied through changing the initial SR picture. The flowchart depicts loading lower-resolution (LR) frames and performing motion estimation (ME) and motion masking on all frames, prior to entering a frame loop. Within the frame loop high frequency information is extracted from the SR input to boost the low resolution information. Then according to this conventional process, the SR picture is updated for each frame in the loop. After processing all frames in this manner the frame loop ends and processing completes.
The conventional state-of-art SR technique of FIG. 1, and others with similar high frequency processing, are subject to a number of problems. One problem is that as the image is sharpened a significant amount of ringing is induced about the edge areas. This problem arises in response to error propagation combining with the high-frequency contents update, and the problem becomes much worse in response to noisy input sources.
In these current super-resolution techniques a high-resolution picture is created from multiple lower-resolution pictures based on a number of estimate models. These estimate models include the following. (1) Creating models of processes from high-resolution (HR) pictures to low-resolution (LR) pictures; such as utilizing geometrical warp models, blurring models, and decimation models. (2) Specific techniques for each of these models can be utilized for reversing the process. (3) Iteratively updating the super-resolution (SR) picture from the initial guess based on a set of LR pictures. A number of issues arise when utilizing these existing super-resolution techniques, such as the accentuation of noise and ringing artifacts in response to boosting high-frequency picture components, and the difficult of making trade-offs between increasing sharpness and decreasing ringing artifacts.
Accordingly, a need exists for super-resolution techniques which reduce ringing and noise problems while facilitating the selection of tradeoffs between increasing sharpness and decreasing ringing artifacts.